The present invention relates in general to Markov chain Monte Carlo (MCMC) analysis and more specifically to systems, methods and computer implemented tools for performing MCMC simulations.
A Markov chain is a sequence of random variables with a probability of transition from a current random variable to a subsequent random variable that depends only on the value (or state) of the current random variable. To generate a realization from a Markov chain, an initial state is either specified or randomly selected, e.g., from a probability distribution over the possible states of the initial random variable. Sampling then commences by building a succession of realizations where the probability density over the possible states of the next realization depends only upon the state of the current realization. A Markov chain is said to have a limiting distribution (also called invariant or stationary distribution) if, as the number of samples in the Markov chain becomes large, the relative frequency of the realizations of the chain uniformly converges to the distribution.
In general terms, a Monte Carlo method is any method that solves a problem by generating suitable random numbers. Thus, Markov chain Monte Carlo (MCMC) simulations provide a means of creating a Markov chain with a given invariant distribution by generating suitable random numbers. Accordingly, if there is a need to sample randomly from a specific probability distribution, then an MCMC simulation may be used to generate a Markov chain that converges to that probability distribution. Once the Markov chain has (approximately) converged, the subsequent realizations from the generated Markov chain can be used as an approximation of samples drawn from the specific probability distribution of interest.
MCMC simulation is a particularly compelling tool for estimating parameters in complex models having a large number of degrees of freedom where an analytical solution is impractical. For example, in numerical analysis, MCMC simulations may be used for high dimensional integration and for finding the minimum and/or maximum of parameters in a model of interest. MCMC simulations may be used in computer science to study random algorithms, e.g., to study whether an algorithm of interest scales appropriately as the size of the underlying problem increases. In physics, MCMC simulations may be used in statistical mechanics and in analyzing complex dynamical systems. In statistics, MCMC simulations have been increasingly used for exploring posterior distributions of random parameters in hierarchical Bayesian models that are generally too complicated to be studied analytically.